$webwork.htmlEncode($page.space.name) : My Geo follies

I'm doing this entry more for myself than anything else. Hopefully there will be something in here that will help someone else, or at least provide some entertainment. But the bottom line is, there will come a day when I will forget the mistakes that I made, the frustration that I felt and the hard fought battles that I won. This entry will remind me that some things worth learning come with a bit of pain - but that only makes the knowledge acquired more valuable.

I got started with GeoTools because I don't like the Terms of Use associated with all the Navteq sites (this includes Google, Yahoo, MapQuest, etc). I have an open source project (Route Ruler) that needs map images in .jpg, .gif or .png format. All the map sites used to work great for this, even if the Terms of Use weren't "perfect". Now, you're no longer able to right click and "Save Image As", so they're basically useless (yeah, you can do a screen print, but try to explain to users how to do that, and then how to crop out all the other junk).

Anyway, I turned to GeoTools. That's when the adventure began! I wanted to create a simple Swing based application to read in Tiger line files, and generate a map. I even started another open source project in order to get some help. We quickly got a prototype running, and just as quickly hit our first real snag. Version 2.1 of GeoTools has some classes that can be used to build a Swing app. But they're not that mature, and we found that there isn't a way to "group" labels. This lead to the label for each street to be drawn for each segment - in other words, the name was repeated between every intersection.

Version 2.2 includes a "VendorOption" tag that can be included in the SLD. This was exactly what we needed. Unfortunately, getting the Swing app to work with version 2.2 proved very frustrating. I wouldn't say that we're at a dead end, but we're probably not going to mess with it any longer until the swing based tools (the swing-widgets subproject) gets a little more "complete".

In a quest to find an way to offer maps to Route Ruler users, I turned to GeoServer. While creating a web based mapping solution isn't optimal, I figured it wouldn't hurt to learn how that project works. I was thrilled when I generated my first image with the street labels grouped. This seemed like a turning point. But, it didn't take long to run into another challenge - I needed to embed a scale in the map image, and GeoServer didn't seem to have a way to do this.

One of the great things about Open Source is, if you can't find a way to do what you want, you can always use brute force. So, I figure out where the map image was being created, and simply added a scale. Not the best solution, but at least I could check this off my list of requirements (sounds like this feature will be included in version 1.4 - I hope so!).

So, now I've got maps being produced that look really nice. Using MapBuilder, I've got a whole page being built, and I feel like I'm making some progress. But wait a minute - there's no way for users to get the map image off the web page! Right clicks are treated just like a left clicks so users can't save the image to their hard drive (I feel like I'm in Google land). While the MapBuilder architecture is well thought out, if you're not well versed in JavaScript, it's a lot to wade through to find what you're looking for. But again, brute force comes to the rescue:

//In MapContainerBase.js
this.eventHandler=function(ev) {
var processItem = true;
if (window.event)

Unknown macro: { //IE events var p = window.event.clientX - this.offsetLeft + document.documentElement.scrollLeft + document.body.scrollLeft; var l = window.event.clientY - this.offsetTop + document.documentElement.scrollTop + document.body.scrollTop; this.evpl = new Array(p,l); this.eventTarget = window.event.srcElement; this.eventType = window.event.type; if (window.event.button>1) processItem=false; this.altKey = window.event.altKey; this.ctrlKey = window.event.ctrlKey; this.shiftKey = window.event.shiftKey; window.event.returnValue = false; if (processItem) window.event.cancelBubble = true; }

Unknown macro: { processItem=false; }

    if (processItem) this.containerModel.setParam(this.eventType,this);
      return false;
    }

But wait! The map isn't an image - it's a div. So even though I've got right clicks working, the browser doesn't recognize the map as an image. Arghhhh. I thought about hacking the crap out of MapBuilder so that the main map is wrapped in an img tag, but fortunately I didn't need to do this. David Blasby included a script in the distribution that he says "is not great JavaScript" - silly rabbit!

I needed to "untrap" the right clicks again in Dave's script (which took no time compared to what I went through before). There's still a problem if the map is too big for the browser - clicking on the scroll bar is registered as a click on the map leading to some unexpected zooming (I'll eventually get around to fixing this). But all in all, I've now got a pretty good way to display maps.

I had spent some time messing around with SLD files with GeoTools, but none of those files where working with my new data. GeoServer validated the files, and showed me all the problems - believe me, there were plenty. I had some problems with "Or" filters and even posted a message on the GeoServer mailing list (unfortunately, I've changed the SLD so much that it's a moot point now - I hate leaving threads unresolved, but . . .). There were some other "hmmmm" moments as well, but I soon found that having the data in an SQL database was the best thing that could have happened.

When I first worked with SLD files, a friend, Ed Mackenzie, wrote up some files that blew me away. I was setting filters based on "fename" and "fetype" (basically things like if fename like 'I-%', then it must be an interstate or if fetype == 'Hwy' then it's a highway). Ed was using things like:

                 <ogc:Filter>
                    <ogc:And>
                    <ogc:PropertyIsBetween>
                        <ogc:PropertyName>Ccode</ogc:PropertyName>
                        <ogc:LowerBoundary><ogc:Literal>10</ogc:Literal></ogc:LowerBoundary>
                        <ogc:UpperBoundary><ogc:Literal>30</ogc:Literal></ogc:UpperBoundary>
                    </ogc:PropertyIsBetween>
                    <ogc:PropertyIsEqualTo>
                        <ogc:PropertyName>Ctype</ogc:PropertyName>
                        <ogc:Literal>A</ogc:Literal>
                    </ogc:PropertyIsEqualTo>
                    </ogc:And>
                </ogc:Filter>

Ctype = "A"? Ccode>10 && Ccode<30? What the heck? This must be some kind of GIS magic. I never spent the time to try to understand it.

But as I'm going through SLD files for my GeoServer installation, I need these incantations. Just to prove that this is all magic, there isn't even a Ccode or a Ctype field in the database:

The only fields that I "know" are fename and fetype - all the others are filled with more magic numbers. Now, a reasonable person would have simply gone to the census bureau site and looked for an explanation of the fields. Being that I'm going through this during the holidays, I'm at home with only a dial up account and no ink my printer - I figured it was worth a bit of trial and error instead of gauranteed frustration and boredom. Fortuanetly, something told me to start at the bottom!

geodata=# select distinct cfcc2 from tiger_roads;
cfcc2
-------
A1
A2
A3
A4
A5
A6
A7
(7 rows)

geodata=# select cfcc2, fename from tiger_roads where cfcc2 = 'A1';
cfcc2 |          fename
----+-----------------------
A1    | I-84
A1    | I-205
A1    | I-84
A1    | United States Highway 30
A1    | I-84
A1    | I-84
A1    | I-84
A1    | I-84
.......

Yeah!!!! The cfcc2 field is my magic field - and I only have 7 values to work with! Now doing things like this is easy:

        <FeatureTypeStyle>
            <FeatureTypeName>Feature</FeatureTypeName>
            <Rule>
                <Name>roads overview</Name>
                <ogc:Filter>
                    <ogc:PropertyIsEqualTo>
                               <ogc:PropertyName>cfcc2</ogc:PropertyName>
                               <ogc:Literal>A4</ogc:Literal>
                           </ogc:PropertyIsEqualTo>
                 </ogc:Filter>
                <MaxScaleDenominator>100000</MaxScaleDenominator>
                <LineSymbolizer>
                    <Stroke>
                        <CssParameter name="stroke">#ff1206</CssParameter>
                        <CssParameter name="stroke-width">1</CssParameter>
                    </Stroke>
                </LineSymbolizer>
            </Rule>
        </FeatureTypeStyle>

If you're looking for some value from this post - something that will save you time - here it is: I spent a lot of wasted time using the "style.do" page (Welcome->Config->Data->Style) UI to edit my SLD files. While this page is easy to use, and validates the SLD files, it eats up a lot of time (click, pick, click, click, pick . . .). Instead, I (man I wish I could say "quickly", but that would be a lie) realized that the SLD files can be edited in place (data->demo->styles) and then loaded by using the "Load" button (so that's what that does). Using Load takes as long as Apply, but you that's the only thing you have to wait on. You'll know you screwed up the SLD file if your image doesn't render any longer (in which case, it's time to use "style.do" again).

So, I added more layers, and more information in my SLD files. But waiting for GeoServer to reload the configuration files was nothing compared to waiting for the GetMap request to return. I'm talking, go to lunch, read a magazine, get a cup of coffee . . . and still you'll have to wait.

To solve this, I started deconstructing my SLD file. Once I actually timed it, I discovered it took over 2 minutes to render 3 layers. By removing all but one layer, it was clear that there was a single layer causing the slow down. Concentrating on that layer, I discovered that the filters I was proud of figuring out were causing the problem. With a single filter on a single layer, rendering took ~32 seconds. Remove the filter and it's reduced to ~3 seconds.

But I need those filters! I want highways rendered differently than regular streets. What to do, what to do.

I would later see WMS services that included layers with titles like "Highways", "Roads" and the like. That would have made the solution a bit more obvious. At least I didn't try hacking the SLDParser class or something like that! While I tried a couple of different methods for getting multiple tables loaded and recognized by GeoServer, the easiest was to use shp2pgsql to make 3 files with the same data. I renamed the table names of two and then loaded them all into Postgresql. With a couple of delete statements, I had all my road data in 3 separate tables, which make 3 very nice layers, each with their own SLD, and each loading in around 3 seconds. Even after putting all of them together, I was just a bit over 3 seconds. Big improvement over 2 minutes!

With the performance problem solved, it became a lot easier to build the SLD files (though if I had a lot more time, I'd put some effort into building a WYSIWYG SLD builder). And again, I encountered another learning opportunity. When styling roads with multiple lines (one on top of the other), there were lines showing up across the roads. In other words, the line on top was just a bit too short and the line beneath would show - this made intersections look really funky. I vaguely remembered seeing something in a mailing list posting that mentioned this, but couldn't find the message (dial up forces you to solve a lot of problems on your own).

So, here's what you don't want to do in an SLD:
        <FeatureTypeStyle>
            <FeatureTypeName>Feature</FeatureTypeName>
            <Rule>
                <Name>roads detail</Name>
                <MaxScaleDenominator>10000</MaxScaleDenominator>
                <LineSymbolizer>
                    <Stroke>
                        <CssParameter name="stroke">#ff1206</CssParameter>
                        <CssParameter name="stroke-width">5</CssParameter>
                    </Stroke>
                </LineSymbolizer>
                <LineSymbolizer>
                    <Stroke>
                        <CssParameter name="stroke">#ffffff</CssParameter>
                        <CssParameter name="stroke-width">4</CssParameter>
                    </Stroke>
                </LineSymbolizer>
                <LineSymbolizer>
                    <Stroke>
                        <CssParameter name="stroke">#ff1206</CssParameter>
                        <CssParameter name="stroke-width">1</CssParameter>
                    </Stroke>
                </LineSymbolizer>
            </Rule>
        </FeatureTypeStyle>

It's a bit more typing, but this works much better:
        <FeatureTypeStyle>
            <FeatureTypeName>Feature</FeatureTypeName>
            <Rule>
                <Name>roads detail</Name>
                <MaxScaleDenominator>10000</MaxScaleDenominator>
                <LineSymbolizer>
                    <Stroke>
                        <CssParameter name="stroke">#ff1206</CssParameter>
                        <CssParameter name="stroke-width">5</CssParameter>
                    </Stroke>
                </LineSymbolizer>
            </Rule>
        </FeatureTypeStyle>
        <FeatureTypeStyle>
            <FeatureTypeName>Feature</FeatureTypeName>
            <Rule>
                <Name>roads detail</Name>
                <MaxScaleDenominator>10000</MaxScaleDenominator>
                <LineSymbolizer>
                    <Stroke>
                        <CssParameter name="stroke">#ffffff</CssParameter>
                        <CssParameter name="stroke-width">4</CssParameter>
                    </Stroke>
                </LineSymbolizer>
            </Rule>
        </FeatureTypeStyle>
        <FeatureTypeStyle>
            <FeatureTypeName>Feature</FeatureTypeName>
            <Rule>
                <Name>roads detail</Name>
                <MaxScaleDenominator>10000</MaxScaleDenominator>
                <LineSymbolizer>
                    <Stroke>
                        <CssParameter name="stroke">#ff6600</CssParameter>
                        <CssParameter name="stroke-width">1</CssParameter>
                    </Stroke>
                </LineSymbolizer>
            </Rule>
        </FeatureTypeStyle>

For now, that's it with GeoServer. Hopefully you've gotten something out of this. I know the experience has been great for me - I feel much better prepared as I return to GeoTools for some Swing based mapping fun!